PCT/IB05 /ot 15 0 
04. 05.05 




s*ypo mn im\ 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

February 23, 2005 

TfflS IS TO CERTIFY THAT ANNEXED HERETO IS A TRUE COPY FROM 
THE RECORDS OF THE UNITED STATES PATENT AND TRADEMARK 
OFFICE OF THOSE PAPERS OF THE BELOW IDENTIFIED PATENT 
APPLICATION THAT MET THE REQUIREMENTS TO BE GRANTED A 
FILING DATE UNDER 35 USC 111. 



m 



APPLICATION NUMBER: 60/618,688 
FILING DATE: October 14, 2004 



PRIORITY 
DOCUMENT 

SUBMITTED OR TRANSMITTED IN 
COMPLIANCE WITH RULE 17.1(a) OR (b) 



By Authority of the 

COMMISSIONER OF PATENTS AND TRADEMARKS 




ill 



ii 




ro 

09 



C 

cl 



V 



PTO/SB/16{09-04) 
Approved for use through 07/31/2006. OMB 0651-0032 
I. ^ ... „ U.S. Patent and Tfademark Office: U.S. DEPARTMEMT OF COMMERCE 

under the Paperwork Reduction Ad of 1995, no persons are required to respond to a collection of Information unless It displays a valid OMB control mmber 

PROVISIONAL APPLICATION FOR PATENT COVER SHEET 

This is a request for filing a PROVISIONAL APPLICATION FOR PATENT under 37 CFR 1,53fe). 

I Express Mall Label NO. ^5^7424149 ^| o 



Given Name (first and middle Of any]) 



BISWAROOP 



Additional inventors are being named on the 



INVENTORQ) 



Family Name or Surname 



MUKHERJEE 



Residence 
(City and either State or Foreign Country) 



OTTAWA. ONTARIO. CANADA 



ONE 




jseparately numbered sheets attached hereto 



TITLE OF THE IMVENTION fSOO characters max^! 



PAIRWISE INDEPENDENT SCHEDULING FOR NODES WITH A SINGLE 
PARAMETERIZABLE RADIO IN A WIRELESS NETWORK 



Direct all correspondence to: CORRESPONDENCE ADDRESS 

The address conresponding to Customer Number 

OR 



00626 



□ 



Rrm or 

individual Name 

Address ' 



City 



Country 



State 



Zip 



Telephone 



Fax 



ENCLOSED APPLICATIOM PARTS (check all that apply) 



Specmca^on Number of Pages 17 



Drawing(8)Afi/mi}erofS/ieefe 4 

□ 



a CD(s), Number of CDs- 

□ 



Other (specify). 



' Application Data Sheet. See 37 CFR 1 .76 



METHOD OF PAYMENT OF RUNG FEES FOR THIS PROVISIONAL APPLICATION FOR PATENT 



Applicant claims small entity status. See 37 CFR 1.27. 



FlUNG FEE 

Afnniint rft\ 



□ 

^— ^ A check or money order is enclosed to cover the filing ffees. 
CD Payment by credit card. Form PTC)-2038 is attached. 

E] The Director is hereby authorized to charge filing fees or credit any overpayment to Deposit Account Number- 500210 
A duplicative copy of this form is enclosed for fee processir^. 



160.00 



□ 
□ 



X*^ invention was made by an agency of the United States Government or under a contract with an agency of the United States 
Government. No. 



Yes, the name of the U.S. Government agency and the Government contract number are: 



SIGNATURE 



TYPED or PRj 




[3ate October 14, 2004 



John Crsne 



25231 



TELEPHONE 972-685-8442 



REGISTRATION NO. . 
(ifapprofmate) ' " 
Docket Number 16892ROUS02P 



^ v. « . ^SE ONLY FOR FlUNG A PROVISIONAL APPUCATION FOR PATENT 

Jn^^^Il'i^u^^'l^J^!!^^'^'^ ^ 51: II? infonif^tion Is required to obtain or retain a benefft ty the public which is to Hie (and by the USPTO 
iL^^lfhoSSf ^f "^^"^l^'^ gcvemed by 35.U.S.a 122 and 37 CFR 1.11 and 1.14. This coll^on is^timated to take 8 hour^tolomF^^^^^ 
irictuding gathenng. prepanng, and submittirig the completed appfication form to the USPTQ Time will vary depending upon the individual caZ Aw^m^rk 
on the arnoum oftme you require to complete this form andtor suggestions for reducing tWs burden, should be^ to IheCWef rtonrofonOTi^^ 

ADDRESS. SEND TO: CommJssioner for Patents, P.O. Box 1450, Alexandria. VA 22313-1450. ^r^m^ i i nio 

If you neeaas^stanoe in complete the tbrm, caU l-eOO-PTO-OiSQ and seled option 2, 



Copy provided by USPTO from the IFW Image Database on 02/18/2005 



PROVISIONAL APPUCATION COVER SHEET 
Additional Pago 



PTO/SB/16(09-04) 
Approved for use through 07/31/2006. OMB 0651-0032 
4u^r^ Patent and Trademark Office; U.S. DEPARTME^^• OF COMMERCE 

the Paperwork Reduction Act of 1995, no peisofTs are required to respond to a coHectlon of irtformatton unless it displays a valid OMB control number. 



Named inventor [bISWAROOP MUKHERJEE | Docket Number 16892ROUS02P 



INVENTOR(S)/APPLICANT(S) 


Given Name (first and middle fif anvl) 




Residence 


BILL 
JERRY 


GAGE 
CHOW 


STITTSVILLE. ONTARIO, CANADA 
KANATA. ONTARIO. CANADA 



Number qmp o f qnf 



WARNING: Infbmiation on this form may become public. Credit card infomiab'on should not be 
included on this form. Provide credit card Information and authorization on PTO-2038. 



Copy provided by USPTO from the IFW Image Database on 02/18/2005 



168g2ROUS02P 



UNITED STATES PROVISIONAL 
PATENT APPLICATION 



Biswaroop Mukherjee 
Bill Gage 
Jerry Chow 
INVENTORS 



PAIRWISE INDEPENDENT 
SCHEDULING FOR NODES 
WITH A SINGLE 
PARAMETERIZABLE RADIO IN 
A WIRELESS NETWORK 



Docket No. 7000-365-P 



Copy provided by USPTO from the IFW Image Database on 02/18/2005 



16892ROUS02P 1 

PAIRWISE INDEPENDENT SCHEDULING FOR NODES WITH A SINGLE 
PARAMETERIZABLE RADIO IN A WIRELESS NETWORK 

Background of the Invention 
5 [0001 ] A network where each node shares a single comnfiunlcation resource 
to exchange data between Its (possibly many) neighbors needs to determine 
when and how it would distribute such resources. If the neighbours contend for 
time (e.g. on a node's CPU or shared radio(s)) then the problem is of determining 
the time a node should devote to each of its neighbours so that maximum 

1 0 throughput is achieved. The case of particular Interest here is that of a radio that 
needs to be multiplexed over the many neighbours, where, in order to obtain 
maximum throughput we need to maximize useful time that neighbours have to 
communicate with each other while avoiding collisions. This determination of 
timing and sequence of communication events is referred to here as the 

1 5 scheduling problem. In other words the problem is for each node to create a 
schedule that detemiines when and for how long to exchange data with its 
neighbours avoiding collision. As an example consider Figure 1 which shows a 
network where each node has a directional radio that it multiplexes over its 
various links. 

20 

Qualifiers 

[0002] Widespread nodes: One reason to create a wireless mesh network is 
to provide widespread coverage by means of wireless nodes that fomi the 
backhaul infrastructure. This is an attempt to extend the model of the model of 

25 wired node access points where backhaul is over the wired network. In order to 
be successful the nodes are expected to be spread out in space, with the 
obvious requirement that at least one node connected to the existing mesh be 
able to communicate with a new outlying node. So the scheduling solution cannot 
rely on all the nodes being able to "hear" schedule announcements from one 

30 source. See for example Figure 1 where nodes C and I extend the reach of the 
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Wired access points A and B. As will be apparent to one of skill In the art, the 
signaling from A or B will not reach all nodes. 

[0003] Directional radios: Directional radios allow the medium to be separated 
into smaller contention zones (as compared to non-directional antennas). On 
5 one hand this can potentially lead to higher throughput if the scheduler can 
properly utilize the smaller contention regions, but on the other hand it 
complicates scheduling and collision avoidance, which nominally relies on 
mechanisms which rely on all nodes that may interfere with a receiver being able 
to hear a clear to send signal (like 802.1 1 IBSS mode), or on schedule 

1 0 announcements from a central controller (like 802.1 1 BSS mode). 

[0004] For example, consider Figure 2 where nodes A and B start to 
communicate with their antennas aligned. If we were using omni antenna, then 
only A and B could have communicated and C and D would have sensed the 
medium to be busy and stay idle. With directional antennas one could hope to 

15 do better. However with directional antennas C cannot sense that the medium 
near B may be busy and send to B Interfering with its reception of packets from 
A. 

[0005] The method proposed does not require scheduling signals to be 
duplicated - each node signals each of its neighbors exclusively, thereby 

20 exploiting directional nature of the antenna. 

[0006] - Independent clocks and synchronization: Creating a schedule will 
however not guarantee that it is followed. It is a well known problem of 
distributed scheduling that the notion of time in different nodes, even If they have 
the same hardware, is non-uniform and varies with time. So following a pre- 

25 determine schedule will fall because of the relative drift of timing references In 
nodes. 

[0007] Some form of periodic synchronization of clocks is required to deal with 
the drift between nodes. In several networks periodic messages are sent from a 
designated node as a reference to which others need to synchronize their time, 
30 (e.g. beacons in 802.1 1 , heartbeats etc.). 
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[0008] However in systems with widespread nodes it is clear that no one node 
can reach all other nodes and convey time at one instant. Even if the nodes 
were close to each other they may not be able to hear beacon-like message if 
their antennas are directional. One can attempt to relay a time message, 
5 however, the amount and variation of delay in relaying such a message will lead . 
to inaccuracies, in addition to being complex and expensive. Finally, relaying 
beacon-like messages will likely be unsuccessful due to different degrees of 
variations where if a node synchronizes with one neighbour by adjusting its clock. 
It may in fact be de-synchronizing with another neighbour. Many approaches 

1 0 have been attempted to resolve these problems, including the following. 

[0009] Centralized scheduling: The 802.1 1 MAC [1] has its own scheduling 
algorithms. In the BSS mode of the algorithm, the Hybrid Control Point function in 
the access point determines the schedule based on the requests sent in by the 
stations and conveys this back to them. Each station then follows this schedule 

1 5 until the next time they receive another schedule. This method is only useful 
when all the stations can hear the schedules being broadcast by the AP. In a 
network using direction radios, such schemes will fail because the broadcasts will 
not reach nodes that are out of range or are not along the beam containing the 
transmitted schedules. The method described here however does not depend 

20 upon a single controller or upon a broadcast mode of operation. In fact, as long 
as a new node can communicate with one of the nodes already in the system, 
the new node can be incorporated and will be a part of the distributed schedule 
being maintained through the network. In addition complexity of computing 
schedules at a central point will increase as the number of links and nodes 

25 increases. It is also conceivable that the centralized schedule would need input 
from all the nodes about attributes like data queues, which will likely become 
inaccurate if the system is dynamic. However, these problems are seen to be 
alleviated by distributed scheduling (e.g. the closer decisions are made to the 
node the less error in estimating queue sizes and fewer links to deal with). 

30 [0010] Decentralized scheduling: In 802.1 1 IBSS mode and in the Distributed 
Control Function mode of a BSS, each node independently attempts to send data 
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after sensing its channel. This strategy allows networks with widely separated 
nodes as there is no requirement for the broadcast of a central schedule. As the 
number of nodes and traffic increases on the medium, nodes that sense the 
medium to be busy will perform exponential bacl<-off required for such random 
5 access of a shared medium and thus the throughput of the system will be much 
lower when compared to the method proposed here. 

[001 1 ] Arbitration: Arbitration schemes that attempt to provide coordination 
such that two neighbors can exchange data when they have packets in their 
queues, such as RTS/CTS signaling may alleviate some of the overhead of 

10 exponential back-off. RTS/CTS schemes have been adapted to directional 
antennas [2][3], where parts or all of the signaling is done using directional 
antenna In order to try to encourage multiple simultaneous non-Interfering 
transmissions [4]. However arbitration schemes like RTS/CTS incur costs that 
grow with the number of nodes in the network and setting up synchronization 

1 5 when a data packet to be sent out on a link arrives, may take an arbitrary amount 
of time depending upon the traffic serviced by the neighbors. The method 
proposed here is constant cost (independent of the number of nodes and their 
distribution) and provides bounded response time and does not require any back- 
off. In addition the proposed mechanism exploits the separation of media 

20 provided by directional antenna (and multiplexing other parameters) to obviate 
the RTS/CTS synchronization with a schedule that is guaranteed to be such that 
at no time sending receiving pair non-distinct. Given properly separated media 
this property of the schedule allows for interference less function without 
RTS/CTS or back-off overhead. 

25 [0012] Hierarchical: A master-slave relation like the BSS mode can be 
extended such that master's slave can be a master to other slaves, thereby 
forming a hierarchical tree of nodes [5]. Such a system can prevent back-off 
because schedules for each node will be determined by its corresponding 
master. However, the drawback of such a system is that the root of the whole 

30 tree or roots of sub-trees become points of failure, making all the subtending 
nodes vulnerable. In addition, propagating the schedule from the root to the 
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leaves is likely to be cumbersome and susceptible to clock errors, where the time 
reference in all the nodes will need to be guaranteed to be within bounds for the 
schedule to work. OthenA^ise if a node's time reference deviates. It will, for 
instance, start sending data when it's not supposed to and disrupt the system. 
5 The method described here has no masters that define the schedule for slaves. 
Indeed each node is independent to infer radio parameters to use from its 
environment and create its own schedule. This way, failure in the schedule of a 
node only affects its own schedule and the interaction with its neighbors, while 
interactions of the neighbors with their other peers are not affected. In addition, 
1 0 two peers keep the notion of their relative time within bounds, which incurs much 
less overhead than all nodes synchronizing to a wall clock determined by the root 
of the tree. 

[0013] The method described here does not require £uiy node to ever change 
Its own notion of time, hence avoiding de-synchronization with other nodes as 
15 described above. It relies on relative time intervals, using an arbitrary and 
coarse-grained notion of clock frequency (licks per second"), and attempts to 
bound the variation in such inten/al with each of its neighbors Independently of 
others by periodically redetennining a fresh set of intervals, thereby avoiding drift. 

20 Summary of the Invention 

[0014] The present invention shows a non-interfering schedule of an ad hoc 
network of nodes with parameterizable radio, which can be achieved by nodes 
that independently coordinate to fomi time varying communicating pairs (that are 
guaranteed to be unique) by local message exchange. The present invention as 

25 described herein is a novel distributed algorithm that creates local schedules in 
such a way that the local schedule in each node and its corresponding data 
transmissions over the shared medium are non-interfering with the schedules of 
other nodes and their transmissions. The algorithm works by isolating pairs of 
nodes as communicating units and maintaining uniqueness of such pairs in time. 

30 This along with parameterized radio ensures that non-interfering node pairs in 
proximity as well as throughout the network can use the shared medium to 
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communicate at the same time without interfering with each other, obviate 
previously proposed collision avoidance schemes. 

1001 5] The novelty of this method is the creation of node pairs, which are 
isolated using parameterized radio to simultaneously communicate over the 
5 shared medium and therefore do not need any other access control. In this way, 
It is expected that the nodes will be able to achieve high parallelism in data 
exchange, as well as maintaining robust and scalable scheduling. 
[0016] The basic principles of the method proposed here include: 

• whether a link is to be active at a time or not is determined at the node 
1 0 pair associated with it depending upon locally available free time slots 

• how long a link should exist is preferably determined at the node pair 
associated with it depending upon local data queues 

• the method maximizes utilization of the medium by maximizing the 
number of node pairs that transfer data to each other without 

1 5 interference by creating a suitable schedules while exploiting 

parameter diversity in the pairs 
[0017] Communication at any time can be viewed as bits traveling over a link 
from one node to another. A link therefore defines a node pair. In wired 
networks links are generally constant over time and their permutations are not of 

20 interest. But in the case of a shared radio network the links and therefore the 
node pairs, vary in time and the way they are created detemnines the nature of 
the network and medium utilization. In fact node pairs are naturally formed in 
shared radio netwoiics, in that the basic act of discovering a wireless node fonns 
a node pair. 

25 [0018] The method proposed creates and uses a completely distributed 
schedule. Each node runs a local scheduler that detemnines the timing and 
sequence of its local activities. The collection of all the local schedules forms the 
global networi< schedule, which need not be computed as a whole. Intead of 
creating the schedule at a point and then distributing it to nodes as done in 

30 Hybrid control Point, or indeed sub schedules that can be passed down from 

master nodes to slave nodes, each node's scheduler is solely responsible for itw 
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own schedule. No element of a node's schedule Is computed anywhere but the 
node. The Infomnatlon required to detemnlne the schedule is local to the node or 
its direct neighbour and is therefore likely to be accurate even in dynamic 
systems. 

5 [0019] In general the communication required for determining schedules such 
that a link between two nodes stays up is determination of exchange of the next 
time two neighbours are to communicate and for how long, repeated over time. 
Indeed with this atomic exchange each node can independently construct its own 
schedule. This is also useful since nodes need not communicate with far away 

1 0 nodes, in fact the scheduling messages need not even be relayed. In this way 
the schedule created avoids the errors associated with stale information and 
errors in conveying schedules that a centralized scheduler would encounter. In 
addition failure of any node in a system affects just that node and none other. 
Note that time can be viewed as another means to multiplex the radio along with 

1 5 other radio parameters. 

[0020] Those skilled in the art will appreciate the scope of the present 
invention and realize additional aspects thereof after reading the following 
detailed description of the preferred embodiments in association with the 
accompanying drawing figures. 

20 

Breif Description of the Fioures 

Figure 1 provides a network of nodes having shared radios In accordance with an 
embodiment of the invention 

Figure2 depicts interference associated with directional antennas in accordance 
25 with an embodiment of the invention 

Figure 3 provides a message negotiation in accordance with an embodiment of 
the invention 

Figure 4 shows drift in nodes in accordance with an embodiment of the invention 



1 
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Detailed Description of the Preferred Embodiments 

[0021] The embodiments set forth below represent the necessary information 
to enable those skilled in the art to practice the Invention and illustrate the best 
5 mode of practicing the invention. Upon reading the following description in light 
of the accompanying drawing figures, those skilled in the art will understand the 
concepts of the invention and will recognize applications of these concepts not 
particularly addressed herein. It should be understood that these concepts and 
applications fall within the scope of the disclosure. 

1 0 [0022] Painvise Independent scheduling exploits radios that are 

parameterizable to be non-interfering (e.g. operate in different frequencies and/or 
use beamforming antenna to localize their area of interference) to provide a 
robust, scalable (in the number of nodes and distance) and adaptive distributed 
scheduling mechanism at a low, fixed overhead cost of periodic coordination 

15 between neighboring node pairs. 

Local schedulino to achieve robust inexpensive Global schedulina 
[0023] Instead of creating a schedule that encompasses all nodes at once 
(e.g. a global schedule), or a cascading schedule determined by a local master 
for Its slaves, the method described here decouples the network's schedule into 

20 independent schedules for neighboring node pairs. Each node is responsible for 
its own schedule. The synchronization is between two neighbors and is 
independent of their synchronization with other neighbors. This allows nodes to 
form pairs that can communicate at the same time without collisions by exploiting 
the separation of the shared media afforded parameterizable radio. The schedule 

25 created is guaranteed to be such that at any time each sender-receiver pair is 
globally distinct in its constituent nodes from all other pairs, and by choosing 
suitable parameters for its radio each pair is made locally non-colliding (the same 
parameters can be used by other nodes that may be far away). The exact 
algorithm to choose such parameters is not described here. 

30 

Localized contention regions 
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[0024] The method described herein creates a schedule that is guaranteed to 
be such that at any time each sender receiver pair is globally distinct in its 
constituent nodes from all other pairs. In other words at any time no node will 
attempt to send to a node that some other node is already sending to. Therefore, 
5 there is no chance of collisions where more than one node sends to the same 
receiver. In addition the shared media is divided into localized non-interfering 
contention regions by choosing parameters for its radios such that no other 
nearby node is Interfered with (however the same parameters can be used by 
other nodes that may be far away). This prevents collisions at a receiver due to 
10 a sender that did not Intend it as a recipient. Together, the schedule and 

parameterized radio therefore provide a network in which each pair operates In 
its contention-free regions that are separated spatially, temporally or by radio 
parameters. 

15 Independence from shared clocks ourelv Inten/al based scheduling 

[0025] PainMse independence in scheduling allows each node to maintain its 
own clock Independently I.e., it doesn't need to change it's own clock to match 
the clock of any other node (e.g. master). The schedule of communication 
between two nodes works on time Intervals, not external time or wall clocks. The 

20 start time and duration of the intervals are negotiated between the neighbors as 
required. They then independently count down to the event and if their clocks are 
within error bounds useful information exchange can occur. Note that this Inten/al 
is such that two neighbors can synchronize to without altering their wall clocks. 
Each node also ensures that its previous commitments are not changed (due to 

25 any extemal synchronization or reset of wall clocks,) thereby atomlcally ensuring 
that subsequent commitments do not conflict with previous ones, thus providing 
non-preemptive scheduling. 

[0026] As an example of painwise independent scheduling In the context of 
Figure 1, all nodes would be able to figure out their own schedules independently 
30 and the signaling will be such that each node only signal Its immediate neighbour 
(e.g. H need only signal G - the node nearest to it). Signalling and sending data 
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to the nearest neighbours also minimizes interference and allows for better 
sharing of the medium. For example in Figure 1 , if H signals to C then its unlikely 
that the link G<-> D can function at the same time since G will experience 
interference. 

5 [0027] With directional antennas existing medium arbitration methods do not 
work, so the schedule is created in a way that pairs are always guaranteed to be 
unique. That is at any time if the schedule in a node pair indicates that they are 
to communicate then no other node's schedule will have an entry for 
communicating with either node in the pair. For example in Figure 1 , the method 

1 0 proposed creates a schedules where C<->B collisions can never happen and the 
pairs of A<->B can communicate at the same time as the pair C<->D without 
Interfering with each other. Such multiplexing yields even more benefits in larger 
networks as seen in the previous figure where many pairs (e.g. A<->C, F<->l, 
D<->E, G<->H, K<->B) can exist as opposed to much fewer such pairs if proper 

1 5 scheduling was not done or if omnI antennae were used. In addition signaling is 
only between node pairs such that omni directional broadcasts or related signals 
are not required. 

[0028] We use the following model to define the problem setting. There exists 
a set of nodes in the network N = {no, ni , nij. Each node m can detennine a 

20 set of it's neighbors, i.e. nodes that are first degree peers Nb,- = {np , rig ...} that it 
can communicate directly with. Upon initialization a node will attempt to gather 
information about its peers and try to communicate with them in order to create 
its neighbor set. The specifics of such a method to discover neighbors is beyond 
scope. We assume that after a node pair has established themselves as 

25 neighbors (i.e. added themselves to the others neighbor set) the two will be able 
to communicate for some time sufficient to exchange a small constant number of 
messages. 

[0029] Time in a node is a non-decreasing counter, which can be read at any 
instant in a variable T. This counter can be viewed as slotted, where a time slot is 
30 an atomic time unit for the scheduler (i.e. a singular useful function can be 

accomplished in a time slot). A node's schedule is then a set of contiguous slots 
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SC = {So , Si.Sz, Sk} = {to ,ti,t2 ... , }, where to is the start of the next slot as 
well as the end of the present slot; ti is the end of slot 0 and start of slot 1 and so 
on and slot S/ occurs before Sj implies that the corresponding start time values 
are such that f/ < tj. Furthermore the schedule only has entries for the finite future 
5 (i.e., tk is a finite value of the time counter) and the current value of Tis never 
greater than to. 

[0030] Each node creates a schedule similar to the one above, such that, if in 
a node a?/ the schedule for any time instant indicates that it is to communicate 
with a neighbor rij then there is a corresponding entry for the same time instant in 
1 0 the schedule of at, that indicates that it is to communicate with n/. This is called 
the exclusive pair condition. If time was denoted in the schedule in terms of 
contiguous discrete intervals, called slots, then the exclusive pair condition 
Implies that if in a node ni, the schedule for a time slot indicates that it is to 
communicate with a neighbor nythen there is a corresponding entry for the same 

1 5 time slot in the schedule of Pj that indicates that it is to communicate with ni. Such 
time slots are represented as Si = {t, , n,, pi ,c,}, where f/ is its start time i.e., the 
value of Tat which slot / is to start, and n, is the corresponding neighbor, p, a 
tuple of radio parameters that is to be used for the slot and c, a list of nodes that 
had contended or may contend on this slot. Note that /?/, p, ,c/ = nullior the 

20 times when there are no neighbors to communicate with. 

[0031] When a new node starts, its scheduler will have slots with no 
neighbors. To begin with discovery of other nodes need not be coordinated in 
time and may happen by random overlap of radio parameters and time. 
However, once discovered neighbours exchange messages and resen/e slots in 

25 each others schedules, allowing subsequent scheduled negotiations to be 

coordinated in time. Since the resen/ations are painwise, if each node resen/es 
slots in a serialized fashion and ensure non-preemption in its schedule it can 
satisfy the exclusive pair condition. This implies that the schedule created would 
have successfully created as many different pairs as possible thereby 

30 maximizing medium usage, given that the radio parameters of two links do not 
overlap in space or time. 
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Negotiating 

[0032] Since tlie slot will begin a finite amount of time after tlie exchange to 
reserve tlie slot, a function tliat computes tiie expected data to be exclianged 
5 using the present queue length and history is used at the time of negotiating the 
slot size. By using suitable policies, such as capping the maximum slot that a 
neighbour is assigned and each busy neighbour is given equal chance to book 
slots, nodes can ensure that their schedule is fair. Variations of booking 
schemes, such as time proportional to the ratio of data expected on one queue to 
1 0 data on all the queues, can be adopted according to the particular application to 
tailor how the neighbours are allocated time. A generalization of the booking 
function is as follows: 

Bpq= { Sfi Si \s a slot in the future that is booked for data transfer from 
node p to q} and can be computed by the function I3pq (.) . . .(2) 

^ ^ ^PQ - ^pq (Qp9. Fpq. f^qp. ^'pq. ^'qp. Opq, Qqp, Ppq) 

where: 

qpq = the queue size for data from node p to qr 

Fpq = { s, : Si is a slot in the future that is available to be booked for data 
transfer from node p to q] 

20 B'= { Si : Si is a slot in the past that was booked for data transfer from node 

pXoq} 

Opq. = i^pq. • c/pq 's the Change in the queue size for data from node p to g 
in the slot /} 

Ppq = {a tuple of radio parameters available for the slots for data transfer 
25 from node p to g} 

[0033] A message exchange sequence is used to negotiate the free slots and 
the time required for the nodes to drain their data. Negotiation between two 
nodes A and B is as follows. As shown in Figure 3 first A notes its clock value at 
30 present as a reference time for the present negotiation and computes the tuple 
{qAB. Fab. B'ab. Qab. Pab) where the elements of the tuple are defined above. A 
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ensure that all time values In the message is relative to the reference time. This 
message Is sent to B which first computes its own tuple {qeA. Fba, B'ba. Qba. Pba} 
with its own reference time. Then the booking functions for the directions AB and 
BA are computed by matching the free slots of both A and B, to find common 
5 time Inten/ais, i.e. slots (or parts of slots), from Fab and Fba until all the free slots 
are exhausted or time for all qab and qba has been booked. Also the argument 
for radio parameter space is used to pick suitable radio parameters. The locally 
available parameter space is used to pick suitable radio parameters. The locally 
available parameter space can be maintained by modules that may scan the 

1 0 environment in idle time. The mechanisms for determining the parameter and 
setting them are dependent upon the particular radios and are therefore excluded 
from this discussion. The resulting set of slots, Bab and BaAexe said to be the set 
of booked slots for that negotiation and are conveyed back to node A so that it 
can book those slots in its schedule as well. 

1 5 [0034] It is possible that tiiere are no parameters that can be used to avoid 
interference, in which case a node (e.g. in Figure 2) (B) may be able to note the 
interfering node (C) and the upcoming slots that it may likely get interfered within 
in susceptible slots I(bc). also marks the present link as Interfered link. The next 
time It negotiates with the interfering node it can convey the interfered slot and 

20 the susceptible slots as well as the radio parameters it can use in those slots. 
The interfering nodes can try to switch the radio parameters it can use in those 
slots. The interfering node can try to switch the radio parameters of its interfering 
link, given radio parameters of the colliding pair. If this fails, the interfering node 
marks its interfering links as an Interfered link and associated slots as susceptible 

25 slots. Unless parameters are available in the future to make the links non- 
interi'ering again the markings are retained and new slots are also marked as 
such. Essentially the pairs now need to ^are the medium. The Interfering node 
can determine if any collisions are possible by comparing its own susceptible 
slots with the susceptible slots sent over to see if there is any time when the 

30 interfering pairs may be active at the same time. From then on the interfering 
pair can book slots avoiding the ones that the colliding pair has booked, to be 
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skewed in time and ttius siiare tlie iink. Figure 3 shows tlie excliange of collision 
information and susceptible slots as well as a general avoidance function that 
works as above. 

[0035] Since Bab and Bba are computed out of Fab and Fba are only one 
5 booking occurs and completes at any given time, the bookings can be 

guaranteed to be serialized and thus to not conflict with any existing bookings. 
Note that in order to convey free slots the nodes need to use relative time, 
because they have free running independent clocks. Relative time is calculated 
by a common event, referred to as the reference point of the negotiation. The 

1 0 further the reference point of B is from that of A, the more error will occur In the 
times booked at both ends (note that we use the error term e In calculating the 
impact of drift as shown In equation (1) - see below). One such reference point 
could be the reception of >l's message at B and If the MAC layer provides 
message acknowledgement the reference points for botii A and B can be very 

1 5 close together in temns of absolute time. Also note that in the above discussion 
Instead of sending the amount of data in queue time measures to drain the 
queue can be directly communicated and used as arguments to the function (2). 
[0036] A given node can repeat this process periodically with all of its 
neighbours to continue to communicate with them. The message exchange 

20 above can be interleaved with user data exchange (e.g. as a preamble, or post- 
amble). However scheduled communication with a neighbour may be lost if no 
booking for a future slot exists in the upcoming schedule. In order to prevent this 
from happening the exchange to future slots books, will in the absence of any 
data to be exchanged, book small slots that will be purely for the sake of 

25 completing another negotiation and thereby keeping the neighbour in the 
schedule. 

Nodes with multiole sharable radios 

[0037] Although the treatment of the scheduling problem above assumes that 
30 nodes have a single radio to multiplex, the solution proposed above can be easily 
extended to nodes with more than radios. The problem is easier if the number of 
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radios fs greater than the number of neighbours that a node has to fomri a link 
with. If however the number of radios is less than the number of linlcs then the 
solution above can be applied as such: Each radio will have its own schedule 
which is maintained independent of the other schedules in the same node, 
5 except during negotiations. When the set of available slots it being calculated, 
slots on all available radios are included. The booking function thus can pick 
slots of any of the neighbours or its own radio's schedules. If more than one 
radio Is available for a given time the function can be clever to select the previous 
radio to minimize radio set up overhead. Indeed the equation (2) and its 

1 0 arguments will still hold except that the slots will have a superscript to denote the 
radio number as s*, where k is the radio identifier and / a slot identifier. However 
an important distinction with the single radio case is that nodes with multiple 
radios need to ensure the serialization of its negotiations and therefore Its slot 
bookings. This can be easily achieved by ensuring that the negotiation with a 

1 5 peer doesn't start until the time that no other negotiation is ongoing. Altematlvely 
If an implementation is such that this condition is too restrictive then the slots can 
be guarded by means of semaphores. For instance the negotiation process can 
mark the slots that it has sent to the other node Fab as under-negotiation and 
other negotiations will then avoid slots that are marked such. If some of these 

20 under-negotiant slots end up being unused they can be marked as free again. 

Dealino with clock drift 

Furthermore in order to avoid the problem of clock drift the method proposed only 
used time intervals that can be bounded as a function of the maximum drift in 

25 individual clocks. Scheduling proposed here relies on time intervals relative to 
each neighbour. Given the woret case variation in the clocks the scheduler is 
able to bound the variation in each interval with each of its neighbours 
independent of others. By periodically detennining a fresh set of inten/als it 
avoids losing synchronization. Indeed the method described here doesn't require 

30 any node to ever change its own notion of time, hence avoiding de- 
synchronization with other nodes. In fact one can compute the maximum time 
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before two nodes must negotiation anotlier interval. Consider the Figure 4, 
wliere a message is sent from B to A. If e is tlie error In timing on the message 
exchange and iriu and nius are deemed to be the minimial slot size and the 
minimal sized slot that can accomplish useful data exchange then: 

5 

mus^ (D + mu) -(e + D + 5D) 
D<, (mus- rriu - e)/5 

10 Where: 

D= maximum time before which two nodes must negotiation another 
interval ^ D 

5 s maximum time that the clocks on two nodes can vary In one unit of 

time 

1 5 [0038] The mechanism above allows nodes to schedule data exchange with 
its neighbors, who will in tum be able to relay data to their neighbors and so forth, 
in this way, a large network can globally exchange data, hop-by-hop, by using a 
completely locally determined scheduler, where locality is a node and its direction 
neighbors. This method is independent of any global clocks and does not require 

20 communication with any entity other than direct neighbors. Further this 

exchange is resilient of clock drifts and avoids interference due to directional 
antennas. 

[0039] Those skilled in the art will recognize improvements and modifications 
to the preferred embodiments of the present invention. All such improvements 
25 and modifications are considered within the scope of the concepts disclosed 
herein. 
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Figure 1 
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Figure 2 
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Figure 3 
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Figure 4 
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